Fly-scan high-throughput coded ptychographic microscopy via active micro-vibration and rolling-shutter distortion correction.

Recent advancements in ptychography have demonstrated the potential of coded ptychography (CP) for high-resolution optical imaging in a lensless configuration. However, CP suffers imaging throughput limitations due to scanning inefficiencies. To address this, we propose what we believe is a novel 'fly-scan' scanning strategy utilizing two eccentric rotating mass (ERM) vibration motors for high-throughput coded ptychographic microscopy. The intrinsic continuity of the 'fly-scan' technique effectively eliminates the scanning overhead typically encountered during data acquisition. Additionally, its randomized scanning trajectory considerably reduces periodic artifacts in image reconstruction. We also developed what we believe to be a novel rolling-shutter distortion correction algorithm to fix the rolling-shutter effects. We built up a low-cost, DIY-made prototype platform and validated our approach with various samples including a resolution target, a quantitative phase target, a thick potato sample and biospecimens. The reported platform may offer a cost-effective and turnkey solution for high-throughput bio-imaging.

Sex differences in zymosan-induced behavioral visceral hypersensitivity and colorectal afferent sensitization.

Sex differences in visceral nociception have been reported in clinical and preclinical studies, but the potential differences in sensory neural encoding of the colorectum between males and females are not well understood. In this study, we systematically assessed sex differences in colorectal neural encoding by conducting high-throughput optical recordings in intact dorsal root ganglia (DRGs) from control and visceral hypersensitive mice. We found an apparent sex difference in zymosan-induced behavioral visceral hypersensitivity: enhanced visceromotor responses to colorectal distension were observed only in male mice, not in female mice. In addition, a higher number of mechanosensitive colorectal afferents were identified per mouse in the zymosan-treated male group than in the saline-treated male group, whereas the mechanosensitive afferents identified per mouse were comparable between the zymosan- and saline-treated female groups. The increased number of identified afferents in zymosan-treated male mice was predominantly from thoracolumbar (TL) innervation, which agrees with the significant increase in the TL afferent proportion in the zymosan group as compared with the control group in male mice. In contrast, female mice showed no difference in the proportion of colorectal neurons between saline- and zymosan-treated groups. Our results revealed a significant sex difference in colorectal afferent innervation and sensitization in the context of behavioral visceral hypersensitivity, which could drive differential clinical symptoms in male and female patients.NEW & NOTEWORTHY We used high-throughput GCaMP6f recordings to study 2,275 mechanosensitive colorectal afferents in mice. Our results revealed significant sex differences in the zymosan-induced behavioral visceral hypersensitivity, which were present in male but not female mice. Male mice also showed sensitization of colorectal afferents in the thoracolumbar pathway, whereas female mice did not. These findings highlight sex differences in sensory neural anatomy and function of the colorectum, with implications for sex-specific therapies for treating visceral pain.

Unraveling Anisotropic and Pulsating Etching of ZnO Nanorods in Hydrochloric Acid via Correlative Electron Microscopy.

Despite much technical progress achieved so far, the exact surface and shape evolution during wet chemical etching is less unraveled, especially in ionically bonded ceramics. Herein, by using in situ liquid cell transmission electron microscopy, a repeated two-stage anisotropic and pulsating periodic etching dynamic is discovered during the pencil shape evolution of a single crystal ZnO nanorod in aqueous hydrochloric acid. Specifically, the nanopencil tip shrinks at a slower rate along [0001̅] than that along the ⟨101̅0⟩ directions, resulting in a sharper ZnO pencil tip. Afterward, rapid tip dissolution happens due to accelerated etching rates along various crystal directions. Concurrently, the vicinal base region of the original nanopencil tip emerges as a new tip followed by the repeated sequence of tip shrinking and removal. The high-index surfaces, such as {101̅m} (m = 0, 1, 2, or 3) and {21̅ 1̅n} (n = 0, 1, 2, or 3), are found to preferentially expose in different ratios. Our 3D electron tomography, convergent beam electron diffraction, middle-angle bright-field STEM, and XPS results indicate the dissociative Cl- species were bound to the Zn-terminated tip surfaces. Furthermore, DFT calculation suggests the preferential Cl- passivation over the {101̅1} and (0001) surfaces of lower energy than others, leading to preferential surface exposures and the oscillatory variation of different facet etching rates. The boosted reactivity due to high-index nanoscale surface exposures is confirmed by comparatively enhanced chemical sensing and CO2 hydrogenation activity. These findings provide an in-depth understanding of anisotropic wet chemical etching of ionic nanocrystals and offer a design strategy for advanced functional materials.

Spatial- and Fourier-domain ptychography for high-throughput bio-imaging.

First envisioned for determining crystalline structures, ptychography has become a useful imaging tool for microscopists. However, ptychography remains underused by biomedical researchers due to its limited resolution and throughput in the visible light regime. Recent developments of spatial- and Fourier-domain ptychography have successfully addressed these issues and now offer the potential for high-resolution, high-throughput optical imaging with minimal hardware modifications to existing microscopy setups, often providing an excellent trade-off between resolution and field of view inherent to conventional imaging systems, giving biomedical researchers the best of both worlds. Here, we provide extensive information to enable the implementation of ptychography by biomedical researchers in the visible light regime. We first discuss the intrinsic connections between spatial-domain coded ptychography and Fourier ptychography. A step-by-step guide then provides the user instructions for developing both systems with practical examples. In the spatial-domain implementation, we explain how a large-scale, high-performance blood-cell lens can be made at negligible expense. In the Fourier-domain implementation, we explain how adding a low-cost light source to a regular microscope can improve the resolution beyond the limit of the objective lens. The turnkey operation of these setups is suitable for use by professional research laboratories, as well as citizen scientists. Users with basic experience in optics and programming can build the setups within a week. The do-it-yourself nature of the setups also allows these procedures to be implemented in laboratory courses related to Fourier optics, biomedical instrumentation, digital image processing, robotics and capstone projects.

Image-free active autofocusing with dual modulation and its application to Fourier single-pixel imaging.

Autofocusing is widely used in applications where sharp image acquisition or projection is needed. Here we report an active autofocusing method for sharp image projection. The method works with wide-field structured illumination and single-pixel detection. To find the focus position, the method illuminates the target object with a set of 3-step phase-shifting Fourier basis patterns repeatedly and collects the backscattered light by using a single-pixel detector through a grating. Dual modulation-dynamic modulation by the time-varying structured illumination and static modulation by the grating-embeds the depth information for the target object in the resulting single-pixel measurements. As such, the focus position can be determined by recovering the Fourier coefficients from the single-pixel measurements and searching for the coefficient with the maximum magnitude. High-speed spatial light modulation not only enables rapid autofocusing but also makes the method work even when the lens system is in continuous motion or the focal length of the lens is continuously adjusted. We experimentally validate the reported method in a self-built digital projector and demonstrate the application of the method in Fourier single-pixel imaging.

Optical ptychography for biomedical imaging: recent progress and future directions [Invited].

Ptychography is an enabling microscopy technique for both fundamental and applied sciences. In the past decade, it has become an indispensable imaging tool in most X-ray synchrotrons and national laboratories worldwide. However, ptychography's limited resolution and throughput in the visible light regime have prevented its wide adoption in biomedical research. Recent developments in this technique have resolved these issues and offer turnkey solutions for high-throughput optical imaging with minimum hardware modifications. The demonstrated imaging throughput is now greater than that of a high-end whole slide scanner. In this review, we discuss the basic principle of ptychography and summarize the main milestones of its development. Different ptychographic implementations are categorized into four groups based on their lensless/lens-based configurations and coded-illumination/coded-detection operations. We also highlight the related biomedical applications, including digital pathology, drug screening, urinalysis, blood analysis, cytometric analysis, rare cell screening, cell culture monitoring, cell and tissue imaging in 2D and 3D, polarimetric analysis, among others. Ptychography for high-throughput optical imaging, currently in its early stages, will continue to improve in performance and expand in its applications. We conclude this review article by pointing out several directions for its future development.

Depth-multiplexed ptychographic microscopy for high-throughput imaging of stacked bio-specimens on a chip.

Imaging a large number of bio-specimens at high speed is essential for many biomedical applications. The common strategy is to place specimens at different lateral positions and image them sequentially. Here we report a new on-chip imaging strategy, termed depth-multiplexed ptychographic microscopy (DPM), for parallel imaging and sensing at high speed. Different from the common strategy, DPM stacks multiple specimens in the axial direction and images the entire z-stack all at once. In our prototype platform, we modify a low-cost car mirror for programmable steering of the incident laser beam. A blood-coated image sensor is then placed underneath the stacked sample for acquiring the resulting diffraction patterns. With the captured images, we perform blind recovery of the incident beam angle and model different layers of the stacked sample as different coded surfaces for object reconstruction. For in vitro experiment, we demonstrate time-lapse cell culture monitoring by imaging 3 stacked microfluidic channels on the coded sensor. For high-throughput cytometric analysis, we image 5 stacked brain sections with a 205-mm2 field of view in ∼50 s. Cytometric analysis is also performed to quantify the cellular proliferation biomarkers on the slides. The DPM approach adds a new degree of freedom for data multiplexing in microscopy, enabling parallel imaging of multiple specimens using a single detector. The demonstrated 6-mm depth of field is among the longest ones in microscopy imaging. The novel depth-multiplexed configuration also complements the miniaturization provided by microfluidics devices, offering a solution for on-chip sensing and imaging with efficient sample handling.

Optofluidic imaging meets deep learning: from merging to emerging.

Propelled by the striking advances in optical microscopy and deep learning (DL), the role of imaging in lab-on-a-chip has dramatically been transformed from a silo inspection tool to a quantitative "smart" engine. A suite of advanced optical microscopes now enables imaging over a range of spatial scales (from molecules to organisms) and temporal window (from microseconds to hours). On the other hand, the staggering diversity of DL algorithms has revolutionized image processing and analysis at the scale and complexity that were once inconceivable. Recognizing these exciting but overwhelming developments, we provide a timely review of their latest trends in the context of lab-on-a-chip imaging, or coined optofluidic imaging. More importantly, here we discuss the strengths and caveats of how to adopt, reinvent, and integrate these imaging techniques and DL algorithms in order to tailor different lab-on-a-chip applications. In particular, we highlight three areas where the latest advances in lab-on-a-chip imaging and DL can form unique synergisms: image formation, image analytics and intelligent image-guided autonomous lab-on-a-chip. Despite the on-going challenges, we anticipate that they will represent the next frontiers in lab-on-a-chip imaging that will spearhead new capabilities in advancing analytical chemistry research, accelerating biological discovery, and empowering new intelligent clinical applications.

Remote referencing strategy for high-resolution coded ptychographic imaging.

The applications of conventional ptychography are limited by its relatively low resolution and throughput in the visible light regime. The new development of coded ptychography (CP) has addressed these issues and achieved the highest numerical aperture for large-area optical imaging in a lensless configuration. A high-quality reconstruction of CP relies on precise tracking of the coded sensor's positional shifts. The coded layer on the sensor, however, prevents the use of cross correlation analysis for motion tracking. Here we derive and analyze the motion tracking model of CP. A novel, to the best of our knowledge, remote referencing scheme and its subsequent refinement pipeline are developed for blind image acquisition. By using this approach, we can suppress the correlation peak caused by the coded surface and recover the positional shifts with deep sub-pixel accuracy. In contrast with common positional refinement methods, the reported approach can be disentangled from the iterative phase retrieval process and is computationally efficient. It allows blind image acquisition without motion feedback from the scanning process. It also provides a robust and reliable solution for implementing ptychography with high imaging throughput. We validate this approach by performing high-resolution whole slide imaging of bio-specimens.

Full-resolution, full-field-of-view, and high-quality fast Fourier single-pixel imaging.

Fourier single-pixel imaging (FSI) uses Fourier basis patterns for spatial light modulation to acquire the Fourier spectrum of the object image. The object image can be reconstructed via an inverse Fourier transform. However, the Fourier basis patterns are inherently gray scale, which results in the difficulty that the patterns can hardly be generated at a high speed by using a commonly used spatial light modulator-digital micromirrors device. To tackle this problem, fast FSI, which uses upsampled and dithered Fourier basis patterns to approximate the gray scale patterns, has been reported, but the achievable spatial resolution has to be sacrificed in the pattern upsampling process. Here we propose a method that can achieve not only full-resolution but also full-field-of-view and high-quality FSI. The key to the proposed method is to use a new, to the best of our knowledge, error diffusion dithering algorithm combined with two different scanning strategies to generate two sets of binarized Fourier basis patterns for spatial light modulation. As a result, two images with a sub-pixel shift from each other are reconstructed. It results in the final high-quality reconstruction by synthesizing the two images. We experimentally demonstrate the method can produce a high-quality 1024 × 768-pixel and full resolution image with a digital micromirror device with 1024 × 768 micromirrors.

Light microscopy-based elastography for the mechanical characterization of zebrafish somitogenesis.

We evaluated the elasticity of live tissues of zebrafish embryos using label-free optical elastography. We employed a pair of custom-built elastic microcantilevers to gently compress a zebrafish embryo and used optical-tracking analysis to obtain the induced internal strain. We then built a finite element method (FEM) model and matched the strain with the optical analysis. The elastic moduli were found by minimizing the root-mean-square errors between the optical and FEM analyses. We evaluated the average elastic moduli of a developing somite, the overlying ectoderm, and the underlying yolk of seven zebrafish embryos during the early somitogenesis stages. The estimation results showed that the average elastic modulus of the somite increased from 150 to 700 Pa between 4- and 8-somite stages, while those of the ectoderm and the yolk stayed between 100 and 200 Pa, and they did not show significant changes. The result matches well with the developmental process of somitogenesis reported in the literature. This is among the first attempts to quantify spatially-resolved elasticity of embryonic tissues from optical elastography.

Batch-based alternating direction methods of multipliers for Fourier ptychography.

Fourier ptychography (FP) has been developed as a general imaging tool for various applications. However, the redundancy data has to be enforced to get a stable recovery, leading to a large dataset and a high computational cost. Based on the additive property of the optical pupils in FP recovery, we report batch-based alternating direction methods of multipliers (ADMM) for FP reconstruction. The reported scheme is performed by implementing partial updates in sub-problems of the standard ADMM. We validate the reconstruction performance using both simulated and experimental measurements. Compared with the embedded pupil function recovery (EPRY) algorithm, the proposed algorithms can converge faster and produce higher-quality images.

Deep distributed optimization for blind diffuser-modulation ptychography.

Blind diffuser-modulation ptychography has emerged as a low-cost technique for micro-nano holographic imaging, which enables breaking the resolution limit of optical systems. However, the existing reconstruction method requires thousands of measurements to recover object and diffuser profile simultaneously, which makes the data acquisition time-consuming and cumbersome. In this Letter, we report a novel, to the best of our knowledge, blind ptychography technique with deep distributed optimization, termed BPD2O. It decomposes the complicated optimization task into subproblems, then introduces extended ptychographical iterative engine and enhanced network solver to optimize each in a distributed strategy. In this way, BPD2O combines the advantages of both model-driven and data-driven strategies, realizing high-fidelity robust ptychography imaging. Extensive experiments validate that BPD2O can realize better resolution and lead to a reduction of more than one order of magnitude in the number of measurements.

High-throughput digital pathology via a handheld, multiplexed, and AI-powered ptychographic whole slide scanner.

The recent advent of whole slide imaging (WSI) systems has moved digital pathology closer to diagnostic applications and clinical practices. Integrating WSI with machine learning promises the growth of this field in upcoming years. Here we report the design and implementation of a handheld, colour-multiplexed, and AI-powered ptychographic whole slide scanner for digital pathology applications. This handheld scanner is built using low-cost and off-the-shelf components, including red, green, and blue laser diodes for sample illumination, a modified stage for programmable sample positioning, and a synchronized image sensor pair for data acquisition. We smear a monolayer of goat blood cells on the main sensor for high-resolution lensless coded ptychographic imaging. The synchronized secondary sensor acts as a non-contact encoder for precisely tracking the absolute object position for ptychographic reconstruction. For WSI, we introduce a new phase-contrast-based focus metric for post-acquisition autofocusing of both stained and unstained specimens. We show that the scanner can resolve the 388-nm linewidth on the resolution target and acquire gigapixel images with a 14 mm × 11 mm area in ∼70 seconds. The imaging performance is validated with regular stained pathology slides, unstained thyroid smears, and malaria-infected blood smears. The deep neural network developed in this study further enables high-throughput cytometric analysis using the recovered complex amplitude. The reported do-it-yourself scanner offers a portable solution to transform the high-end WSI system into one that can be made widely available at a low cost. The capability of high-throughput quantitative phase imaging may also find applications in rapid on-site evaluations.

Blood-Coated Sensor for High-Throughput Ptychographic Cytometry on a Blu-ray Disc.

The Blu-ray drive is an engineering masterpiece that integrates disc rotation, pickup head translation, and three lasers in a compact and portable format. Here, we integrate a blood-coated image sensor with a modified Blu-ray drive for high-throughput cytometric analysis of various biospecimens. In this device, samples are mounted on the rotating Blu-ray disc and illuminated by the built-in lasers from the pickup head. The resulting coherent diffraction patterns are then recorded by the blood-coated image sensor. The rich spatial features of the blood-cell monolayer help down-modulate the object information for sensor detection, thus forming a high-resolution computational biolens with a theoretically unlimited field of view. With the acquired data, we develop a lensless coherent diffraction imaging modality termed rotational ptychography for image reconstruction. We show that our device can resolve the 435 nm line width on the resolution target and has a field of view only limited by the size of the Blu-ray disc. To demonstrate its applications, we perform high-throughput urinalysis by locating disease-related calcium oxalate crystals over the entire microscope slide. We also quantify different types of cells on a blood smear with an acquisition speed of ∼10,000 cells per second. For in vitro experiments, we monitor live bacterial cultures over the entire Petri dish with single-cell resolution. Using biological cells as a computational lens could enable new intriguing imaging devices for point-of-care diagnostics. Modifying a Blu-ray drive with the blood-coated sensor further allows the spread of high-throughput optical microscopy from well-equipped laboratories to citizen scientists worldwide.

Robust multi-angle structured illumination lensless microscopy via illumination angle calibration.

Multi-angle structured illumination lensless (MASIL) microscopy enables high-resolution image recovery over a large field of view. Successful image recovery of MASIL microscopy, however, relies on an accurate knowledge of the multi-angle illumination. System misalignments and slight deviations from the true illumination angle may result in image artifacts in reconstruction. Here we report a MASIL microscopy system that is robust against illumination misalignment. To calibrate the illumination angles, we design and use a double-sided mask, which is a glass wafer fabricated with a ring-array pattern on the upper surface and a disk-array pattern on the lower surface. As such, the illumination angles can be decoded from the captured images by estimating the relative displacement of the two patterns. We experimentally demonstrate that this system can achieve successful image recovery without any prior knowledge of the illumination angles. The reported approach provides a simple yet robust resolution for wide-field lensless microscopy. It can solve the LED array misalignment problem and calibrate angle-varied illumination for a variety of applications.

Ringing-free fast Fourier single-pixel imaging.

Fourier single-pixel imaging (FSI) allows an image to be reconstructed by acquiring the Fourier spectrum of the image using a single-pixel detector. Fast FSI is typically achieved by acquiring a truncated Fourier spectrum, that is, only low-frequency Fourier coefficients are acquired, with the high-frequency coefficients discarded. However, the truncation of the Fourier spectrum leads to undesirable ringing artifacts in the resulting image. Ringing artifacts produce false edges in the image and reduce the image contrast, resulting in image quality degeneration. The artifact is particularly severe in dynamic FSI, where the sampling ratio is generally ultra-low. We propose an effective and fast deringing algorithm to achieve ringing-free fast FSI. The algorithm eliminates ringing artifacts through 2D sub-pixel shifting and preserves image details through image fusion. Both static and dynamic imaging results demonstrate that the proposed method can reconstruct ringing-free images from under-sampled data in FSI. The deringing algorithm not only provides FSI with the capability of fast high-quality single-pixel imaging but also might prove its applicability in other areas, such as Fourier-based data compression algorithms.

Ptychographic sensor for large-scale lensless microbial monitoring with high spatiotemporal resolution.

Traditional microbial detection methods often rely on the overall property of microbial cultures and cannot resolve individual growth event at high spatiotemporal resolution. As a result, they require bacteria to grow to confluence and then interpret the results. Here, we demonstrate the application of an integrated ptychographic sensor for lensless cytometric analysis of microbial cultures over a large scale and with high spatiotemporal resolution. The reported device can be placed within a regular incubator or used as a standalone incubating unit for long-term microbial monitoring. For longitudinal study where massive data are acquired at sequential time points, we report a new temporal-similarity constraint to increase the temporal resolution of ptychographic reconstruction by 7-fold. With this strategy, the reported device achieves a centimeter-scale field of view, a half-pitch spatial resolution of 488 nm, and a temporal resolution of 15-s intervals. For the first time, we report the direct observation of bacterial growth in a 15-s interval by tracking the phase wraps of the recovered images, with high phase sensitivity like that in interferometric measurements. We also characterize cell growth via longitudinal dry mass measurement and perform rapid bacterial detection at low concentrations. For drug-screening application, we demonstrate proof-of-concept antibiotic susceptibility testing and perform single-cell analysis of antibiotic-induced filamentation. The combination of high phase sensitivity, high spatiotemporal resolution, and large field of view is unique among existing microscopy techniques. As a quantitative and miniaturized platform, it can improve studies with microorganisms and other biospecimens at resource-limited settings.

Quantitative multi-height phase retrieval via a coded image sensor.

Multi-height phase retrieval introduces different object-to-detector distances for obtaining phase diversity measurements. In the acquisition process, the slow-varying phase information, however, cannot be converted to intensity variations for detection. Therefore, the low-frequency contents of the phase profile are lost during acquisition and cannot be properly restored via phase retrieval. Here, we demonstrate the use of a coded image sensor for addressing this challenge in multi-height phase retrieval. In our scheme, we add a coded layer on top of the image sensor for encoding the slow-varying complex wavefronts into intensity variations of the modulated patterns. Inspired by the concept of blind ptychography, we report a reconstruction scheme to jointly recover the complex object and the unknown coded layer using multi-height measurements. With both simulation and experimental results, we show that the recovered phase is quantitative and the slow-varying phase profiles can be properly restored using lensless multi-height measurements. We also show that the image quality using the coded sensor is better than that of a regular image sensor. For demonstrations, we validate the reported scheme with various biospecimens and compare the results to those of regular lensless multi-height phase retrieval. The use of a coded image sensor may enable true quantitative phase imaging for the lensless multi-height, multi-wavelength, and transport-of-intensity equation approaches.

Optofluidic ptychography on a chip.

We report the implementation of a fully on-chip, lensless microscopy technique termed optofluidic ptychography. This imaging modality complements the miniaturization provided by microfluidics and allows the integration of ptychographic microscopy into various lab-on-a-chip devices. In our prototype, we place a microfluidic channel on the top surface of a coverslip and coat the bottom surface with a scattering layer. The channel and the coated coverslip substrate are then placed on top of an image sensor for diffraction data acquisition. Similar to the operation of a flow cytometer, the device utilizes microfluidic flow to deliver specimens across the channel. The diffracted light from the flowing objects is modulated by the scattering layer and recorded by the image sensor for ptychographic reconstruction, where high-resolution quantitative complex images are recovered from the diffraction measurements. By using an image sensor with a 1.85 µm pixel size, our device can resolve the 550 nm linewidth on the resolution target. We validate the device by imaging different types of biospecimens, including C. elegans, yeast cells, paramecium, and closterium sp. We also demonstrate a high-resolution ptychographic reconstruction at a video framerate of 30 frames per second. The reported technique can address a wide range of biomedical needs and engenders new ptychographic imaging innovations in a flow cytometer configuration.